Learning Grounded Relational Symbols from Continuous Data for Abstract Reasoning

Learning from experience how to manipulate an environment in a goal-directed manner is one of the central challenges in research on autonomous robots. In the case of object manipulation, efficient learning and planning should exploit the underlying relational structure of manipulation problems and combine geometric state descriptions with abstract symbolic representations. When appropriate symbols are not predefined they need to be learned from geometric data. In this paper we present an approach for learning symbolic relational abstractions of geometric features such that these symbols enable a robot to learn abstract transition models and to use them for goal-directed planning of motor primitive sequences. This is framed as an optimization problem, where a loss function evaluates how predictive the learned symbols are for the effects of given motor primitives as well as for reward. The approach is embedded in a full-fledged symbolic relational model-based reinforcement learning setting, where both the symbols as well as the abstract transition and reward models are learned from experience. We quantitatively compare the approach to simpler baselines in an object manipulation task and demonstrate it on a real-world robot.